One of the most important main indicators in the performance criteria of self-driving cars is the policy adopted by the self-driving system regarding the determination of vehicle speed and steering angle. To determine this policy, researchers have always faced the challenge of choosing our optimal training method between two traditional modular and modern End-to-End approaches. Recently, a lot of research has been done in order to introduce the End-to-End approach and its application in this field. In this research, an optimal model for predicting the driver's behavior has been presented using this modern approach in the form of deep learning for training artificial neural networks. In other words, achieving a model with acceptable accuracy compared to similar tasks in driving a self-driving car has been considered. For this purpose, based on the investigations carried out on the architecture of the existing networks, two architectures that have the necessary potential to achieve this goal were selected. Also, in order not to ignore the time relationship between the slides to show the visual time dependencies, and to check its effect on the result, the combination of convolutional neural networks (convolutional) with a type of recurrent network called long short-term memory LSTM was used in the training of the model. Also, a complete data set collected in real driving conditions and labeled including images and depth information has been used, and by designing training algorithms and optimizing training parameters using the Adam optimization algorithm Several trained models were presented. Among the obtained results, some predictions were more optimal than similar works, which shows the unique effect of temporal dependencies in the training and effectiveness of recurrent networks along with the strong processing of convolutional networks.